HADOOP-19964. Restore TimedOutTestsListener thread dumps on test timeout. by joseluisll · Pull Request #8682 · apache/hadoop

joseluisll · 2026-08-13T14:14:37Z

Description of PR

JIRA: HADOOP-19964

GenericTestUtils.waitFor built a 16 KB thread dump into every
TimeoutException message, at 650 call sites. It now dumps to stderr the
moment the wait expires, while the threads are still hung, and the message
drops to one line. Sample failure report: 18,223 -> 1,642 bytes.

TimedOutTestsListener has registered nothing since HADOOP-19415 Part 4 moved
Surefire onto the JUnit Platform provider, which ignores the listener
property. It now implements TestExecutionListener, so the property the eight
poms already carry binds to it once the provider accepts platform listeners
(SUREFIRE-1639, apache/maven-surefire#3438). Dormant until then. No pom is
touched.

-Dhadoop.test.timedout.dump=false disables dumps.
-Dhadoop.test.timedout.dump.limit (default 5) caps them per JVM. Both entry
points share the budget.
The message records "Thread dump printed to stderr." only when one was.

Not covered: a fork killed at forkedProcessTimeoutInSeconds (Shutdown.KILL
-> Runtime.halt()). HADOOP-19950 captures those.

Release note: waitFor's TimeoutException message no longer carries the
thread dump. Nothing in the Hadoop tree asserted on it; downstream code that
does will need adjusting.

Contains content generated by Claude Code.

How was this patch tested?

mvn test -pl hadoop-common-project/hadoop-common -Dtest='TestTimedOutTestsListener,TestGenericTestUtils'
— 15 tests, all pass.

TestTimedOutTestsListener covers deadlock detection, timeout-failure
detection, the off switch, the dump limit and its elision notice,
dumpForTimeout's return value, and waitFor's exact message with a dump
printed, with dumps off, and with the budget spent — driving the real
waitFor. Checkstyle clean on both listener files; the two violations the 2012
test file carried are fixed.

For code changes:

Does the title of this PR start with the corresponding JIRA issue id?
Object storage: integration tests — N/A
New dependencies licensed compatibly with ASF 2.0 — N/A, none added
LICENSE, LICENSE-binary, NOTICE-binary updated — N/A

AI Tooling

The PR includes the phrase "Contains content generated by Claude Code"
My use of AI contributions follows the ASF legal policy
https://www.apache.org/legal/generative-tooling.html

joseluisll · 2026-08-13T14:20:04Z

Cross-reference: this pairs with #8659 (HDFS-17957), which sets a default
junit.jupiter.execution.timeout.default on the hdfs modules.
The two cover different halves of the same problem. This PR restores the thread
dump for tests that fail through JUnit, and explicitly cannot cover a fork
killed at forkedProcessTimeoutInSeconds: Surefire sends the fork
Shutdown.KILL, the fork executes Runtime.halt(), and that bypasses
listeners and shutdown hooks alike. #8659 is what moves hung hdfs tests off
that path — with a default @Timeout they fail through JUnit instead, at which
point this listener produces the dump. Neither PR alone yields a usable
diagnostic for a hung HDFS test.
The two touch one file in common, hadoop-hdfs-project/hadoop-hdfs/pom.xml:
#8659 adds inside <systemPropertyVariables>, this one removes the dead
<properties> listener block just after it. I trial-merged the two heads and
the result is clean, with both changes intact, in either order — no rebase
needed whichever lands first.

pan3793 · 2026-08-19T08:37:33Z

The AI-generated PR description is too verbose

hadoop-yetus · 2026-08-19T18:35:05Z

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	18m 28s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 3 new or modified test files.
			_ trunk Compile Tests _
+1 💚	mvninstall	49m 7s		trunk passed
+1 💚	compile	18m 35s		trunk passed with JDK Ubuntu-21.0.11+10-1-24.04.2-Ubuntu
+1 💚	compile	18m 7s		trunk passed with JDK Ubuntu-17.0.19+10-1-24.04.2-Ubuntu
+1 💚	checkstyle	1m 29s		trunk passed
+1 💚	mvnsite	2m 0s		trunk passed
+1 💚	javadoc	1m 27s		trunk passed with JDK Ubuntu-21.0.11+10-1-24.04.2-Ubuntu
+1 💚	javadoc	1m 24s		trunk passed with JDK Ubuntu-17.0.19+10-1-24.04.2-Ubuntu
+1 💚	spotbugs	3m 12s		trunk passed
+1 💚	shadedclient	37m 2s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	1m 15s		the patch passed
+1 💚	compile	16m 51s		the patch passed with JDK Ubuntu-21.0.11+10-1-24.04.2-Ubuntu
+1 💚	javac	16m 51s		the patch passed
+1 💚	compile	18m 16s		the patch passed with JDK Ubuntu-17.0.19+10-1-24.04.2-Ubuntu
+1 💚	javac	18m 16s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	1m 23s		hadoop-common-project/hadoop-common: The patch generated 0 new + 10 unchanged - 3 fixed = 10 total (was 13)
+1 💚	mvnsite	1m 57s		the patch passed
+1 💚	javadoc	1m 20s		the patch passed with JDK Ubuntu-21.0.11+10-1-24.04.2-Ubuntu
+1 💚	javadoc	1m 18s		the patch passed with JDK Ubuntu-17.0.19+10-1-24.04.2-Ubuntu
+1 💚	spotbugs	3m 22s		the patch passed
+1 💚	shadedclient	37m 42s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	23m 26s		hadoop-common in the patch passed.
+1 💚	asflicense	1m 15s		The patch does not generate ASF License warnings.
		260m 15s

Subsystem	Report/Notes
Docker	ClientAPI=1.55 ServerAPI=1.55 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8682/3/artifact/out/Dockerfile
GITHUB PR	#8682
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname	Linux 0d6d2dafd7bb 5.15.0-181-generic #191-Ubuntu SMP Fri May 22 19:09:02 UTC 2026 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `d2d6f0b`
Default Java	Ubuntu-17.0.19+10-1-24.04.2-Ubuntu
Multi-JDK versions	/usr/lib/jvm/java-21-openjdk-amd64:Ubuntu-21.0.11+10-1-24.04.2-Ubuntu /usr/lib/jvm/java-17-openjdk-amd64:Ubuntu-17.0.19+10-1-24.04.2-Ubuntu
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8682/3/testReport/
Max. process+thread count	1557 (vs. ulimit of 10000)
modules	C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8682/3/console
versions	git=2.43.0 maven=3.9.15 spotbugs=4.9.7
Powered by	Apache Yetus 0.14.1 https://yetus.apache.org

This message was automatically generated.

pan3793 · 2026-08-21T08:38:41Z

The Maven community has accepted SUREFIRE-1639, and it will be included in the 3.6.0-M2. Let me convert this to a draft first and wait for the release.

joseluisll · 2026-09-03T22:25:49Z

The Maven community has accepted SUREFIRE-1639, and it will be included in the 3.6.0-M2. Let me convert this to a draft first and wait for the release.

It has just been released as part of version 3.6.0 here: https://github.com/apache/maven-surefire/releases/tag/surefire-3.6.0

pan3793 · 2026-09-04T07:37:20Z

@joseluisll, but you didn't do the surefire plugin update?

pan3793 · 2026-09-04T07:40:39Z

we may need to wait for 3.6.1 because there is a bug apache/maven-surefire#3453

joseluisll · 2026-09-04T08:13:25Z

@joseluisll, but you didn't do the surefire plugin update?

Nope, will do the plugin update in the POM and check for impact. It seems that the new listener property requires adding a test-scope hadoop-common test-jar dependency for these two modules: hadoop-yarn-services-core, hadoop-yarn-applications-catalog-webapp . Will do a validation for all the modules as well, not only those two.

@timeout

GenericTestUtils.waitFor dumps all threads with deadlock analysis to stderr at the moment the wait expires. Its TimeoutException message drops from ~16KB to one line. Sample failure report: 18,223 -> 1,642 bytes. TimedOutTestsListener implements TestExecutionListener, and Surefire moves to 3.6.0, which accepts platform listeners in the `listener` provider property (SUREFIRE-1639). The property the eight poms already carry now registers it, restoring dumps for @timeout failures across hadoop-common, KMS, HDFS, HTTPFS and all of YARN and MapReduce. hadoop-yarn-services-core and hadoop-yarn-applications-catalog-webapp inherit that property but depended on hadoop-common's main jar only; both now take its test-jar in test scope. Without it Surefire 3.6.0 cannot resolve the listener and the fork dies before any test runs. - -Dhadoop.test.timedout.dump=false disables dumps. - -Dhadoop.test.timedout.dump.limit (default 5) caps them per JVM, with one elision notice. Both entry points share the budget. - The exception records "Thread dump printed to stderr." only when a dump was printed. - The two checkstyle violations the 2012 test file carried are fixed. Not covered: Surefire's fork kill at forkedProcessTimeoutInSeconds (Shutdown.KILL -> Runtime.halt()). HADOOP-19950 captures those reports. Release note: Surefire moves to 3.6.0, and waitFor's TimeoutException message no longer carries the thread dump. Tests: TestTimedOutTestsListener and TestGenericTestUtils pass (15 tests). A hanging @timeout(3) test produces exactly one labelled dump through the listener. Full-tree build passes on 119 modules, and the listener resolves on the test classpath of all 36 modules carrying the property. Contains content generated by Claude Code. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RrG4oLaFUcjSuv9Q5zeaZe

hadoop-yetus · 2026-09-04T12:07:36Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 22s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+0 🆗	xmllint	0m 0s		xmllint was not available.
+1 💚	@author	0m 1s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 3 new or modified test files.
			_ trunk Compile Tests _
+0 🆗	mvndep	1m 51s		Maven dependency ordering for branch
+1 💚	mvninstall	32m 32s		trunk passed
+1 💚	compile	14m 35s		trunk passed with JDK Ubuntu-21.0.12+8-1-24.04-Ubuntu
+1 💚	compile	11m 2s		trunk passed with JDK Ubuntu-17.0.20+8-1-24.04-Ubuntu
+1 💚	checkstyle	3m 0s		trunk passed
+1 💚	mvnsite	3m 8s		trunk passed
+1 💚	javadoc	2m 44s		trunk passed with JDK Ubuntu-21.0.12+8-1-24.04-Ubuntu
+1 💚	javadoc	2m 38s		trunk passed with JDK Ubuntu-17.0.20+8-1-24.04-Ubuntu
+0 🆗	spotbugs	0m 35s		branch/hadoop-project no spotbugs output file (spotbugsXml.xml)
+1 💚	shadedclient	16m 9s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 16s		Maven dependency ordering for patch
+1 💚	mvninstall	1m 42s		the patch passed
+1 💚	compile	8m 17s		the patch passed with JDK Ubuntu-21.0.12+8-1-24.04-Ubuntu
+1 💚	javac	8m 17s		the patch passed
+1 💚	compile	15m 50s		the patch passed with JDK Ubuntu-17.0.20+8-1-24.04-Ubuntu
+1 💚	javac	15m 50s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	2m 55s		root: The patch generated 0 new + 10 unchanged - 3 fixed = 10 total (was 13)
+1 💚	mvnsite	3m 12s		the patch passed
+1 💚	javadoc	2m 42s		the patch passed with JDK Ubuntu-21.0.12+8-1-24.04-Ubuntu
+1 💚	javadoc	2m 38s		the patch passed with JDK Ubuntu-17.0.20+8-1-24.04-Ubuntu
+0 🆗	spotbugs	0m 33s		hadoop-project has no data from spotbugs
+1 💚	shadedclient	16m 11s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	0m 27s		hadoop-project in the patch passed.
-1 ❌	unit	19m 26s	/patch-unit-hadoop-common-project_hadoop-common.txt	hadoop-common in the patch passed.
+1 💚	unit	19m 4s		hadoop-yarn-services-core in the patch passed.
-1 ❌	unit	1m 6s	/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-applications_hadoop-yarn-applications-catalog_hadoop-yarn-applications-catalog-webapp.txt	hadoop-yarn-applications-catalog-webapp in the patch failed.
+1 💚	asflicense	0m 46s		The patch does not generate ASF License warnings.
		197m 23s

Reason	Tests
Failed junit tests	hadoop.http.TestSSLHttpServerConfigs
	hadoop.http.TestHttpCookieFlag
	hadoop.net.TestDNS
	hadoop.http.TestHttpServer

Subsystem	Report/Notes
Docker	ClientAPI=1.56 ServerAPI=1.56 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8682/4/artifact/out/Dockerfile
GITHUB PR	#8682
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets xmllint
uname	Linux 73d6e7910916 5.15.0-190-generic #200-Ubuntu SMP Fri Aug 7 15:06:04 UTC 2026 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `98efd0d`
Default Java	Ubuntu-17.0.20+8-1-24.04-Ubuntu
Multi-JDK versions	/usr/lib/jvm/java-21-openjdk-amd64:Ubuntu-21.0.12+8-1-24.04-Ubuntu /usr/lib/jvm/java-17-openjdk-amd64:Ubuntu-17.0.20+8-1-24.04-Ubuntu
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8682/4/testReport/
Max. process+thread count	1283 (vs. ulimit of 10000)
modules	C: hadoop-project hadoop-common-project/hadoop-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-services/hadoop-yarn-services-core hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp U: .
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8682/4/console
versions	git=2.43.0 maven=3.9.15 spotbugs=4.9.7
Powered by	Apache Yetus 0.14.1 https://yetus.apache.org

This message was automatically generated.

joseluisll · 2026-09-04T12:11:37Z

@pan3793: It seems that the surefire plugin update to version 3.6 breaks the tests (the stack suddenly prefers IPv6 instead of IPv4) . ¿Should I investigate the update of the maven-surefire-plugin in this PR as well? ¿or shall I create a specific JIRA/PR to update maven-surefire-plugin to version 3.6?

github-actions Bot added YARN MapReduce HDFS build trunk Common labels Aug 13, 2026

joseluisll marked this pull request as ready for review August 13, 2026 14:20

joseluisll mentioned this pull request Aug 13, 2026

HDFS-17957. Shut down leaked mini-cluster instances and review timeout values in hadoop-hdfs-project tests. #8659

Merged

6 tasks

joseluisll force-pushed the HADOOP-19964 branch from cb2e328 to 0d7c41b Compare August 15, 2026 05:31

pan3793 reviewed Aug 19, 2026

View reviewed changes

Comment thread hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/test/GenericTestUtils.java Outdated

pan3793 reviewed Aug 19, 2026

View reviewed changes

Comment thread hadoop-common-project/hadoop-common/pom.xml

pan3793 reviewed Aug 19, 2026

View reviewed changes

Comment thread ...ommon/src/test/resources/META-INF/services/org.junit.platform.launcher.TestExecutionListener Outdated

joseluisll force-pushed the HADOOP-19964 branch from 0d7c41b to d2d6f0b Compare August 19, 2026 14:13

github-actions Bot removed YARN HDFS labels Aug 19, 2026

pan3793 marked this pull request as draft August 21, 2026 08:38

joseluisll marked this pull request as ready for review September 3, 2026 22:26

joseluisll requested a review from pan3793 September 3, 2026 22:27

joseluisll force-pushed the HADOOP-19964 branch from d2d6f0b to 98efd0d Compare September 4, 2026 08:49

github-actions Bot added the YARN label Sep 4, 2026

joseluisll marked this pull request as draft September 4, 2026 12:11

Navigation Menu

Uh oh!

HADOOP-19964. Restore TimedOutTestsListener thread dumps on test timeout. - #8682

HADOOP-19964. Restore TimedOutTestsListener thread dumps on test timeout.#8682
joseluisll wants to merge 1 commit into
apache:trunkfrom
joseluisll:HADOOP-19964

joseluisll commented Aug 13, 2026 •

edited

Loading

Uh oh!

joseluisll commented Aug 13, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

pan3793 commented Aug 19, 2026

Uh oh!

hadoop-yetus commented Aug 19, 2026

Uh oh!

pan3793 commented Aug 21, 2026

Uh oh!

joseluisll commented Sep 3, 2026

Uh oh!

pan3793 commented Sep 4, 2026

Uh oh!

pan3793 commented Sep 4, 2026

Uh oh!

joseluisll commented Sep 4, 2026 •

edited

Loading

Uh oh!

hadoop-yetus commented Sep 4, 2026

Uh oh!

joseluisll commented Sep 4, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

joseluisll commented Aug 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description of PR

How was this patch tested?

For code changes:

AI Tooling

Uh oh!

joseluisll commented Aug 13, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

pan3793 commented Aug 19, 2026

Uh oh!

hadoop-yetus commented Aug 19, 2026

Uh oh!

pan3793 commented Aug 21, 2026

Uh oh!

joseluisll commented Sep 3, 2026

Uh oh!

pan3793 commented Sep 4, 2026

Uh oh!

pan3793 commented Sep 4, 2026

Uh oh!

joseluisll commented Sep 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

hadoop-yetus commented Sep 4, 2026

Uh oh!

joseluisll commented Sep 4, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

joseluisll commented Aug 13, 2026 •

edited

Loading

joseluisll commented Sep 4, 2026 •

edited

Loading